Systems and methods for surveillance with a visual marker

ABSTRACT

A method of controlling an unmanned aerial vehicle (UAV) includes receiving an image from one or more vision sensors of the UAV. The image is captured while the UAV is in flight and includes a marker within an environment. The marker has a visual pattern. The method further includes identifying the visual pattern to determine a plurality of instructions encoded in the visual pattern, and controlling, in response to the plurality of instructions, the UAV to perform an action including at least one of adjusting position of a payload, swapping a payload, or capturing an image of a selected subject.

CROSS-REFERENCE

This application is a continuation of application Ser. No. 15/289,384,filed on Oct. 10, 2016, which is a continuation of InternationalApplication No. PCT/CN2014/090078, filed on Oct. 31, 2014. Theabove-referenced applications are hereby incorporated by reference intheir entireties.

BACKGROUND OF THE DISCLOSURE

Aerial vehicles such as unmanned aerial vehicles (UAVs) can be used forperforming surveillance, reconnaissance, and exploration tasks formilitary and civilian applications. Such aerial vehicles may carry apayload configured to perform a specific function.

Typical UAVs may communicate with external systems when navigating anenvironment. A UAV may be operating in an environment in whichnavigation signals such as signals from global positioning softwaresignals (GPS) are weak. This may lead to challenges with autonomousnavigation of UAVs

SUMMARY OF THE DISCLOSURE

A need exists to provide a method of navigating an unmanned aerialvehicle (UAV) in an environment without relying on signal strength andreliability of navigation sensors such as global positioning softwaresignals (GPS). Provided herein are systems and methods for positioningand navigating a UAV in an environment. The systems and methods furtherprovide the ability to communicate directions to a UAV for surveillanceand navigation purposes. Markers may be provided in an environment toaid in UAV navigation. The markers may be visual markers that may permita UAV to get its bearings and/or convey instructions to the UAV.

In an aspect of the present disclosure, a method for conductingsurveillance of an environment comprises positioning an unmanned aerialvehicle (UAV) within the environment, wherein said positioning isdetermined by detecting, using one or more vision sensors of the UAV,one or more visual markers having visual patterns and attached to one ormore surfaces within the environment; and collecting visual imagesand/or audio data within the environment using one or more sensors ofthe UAV, thereby conducting the surveillance of the environment.

The environment can be an indoor environment. The environment can be anenvironment in which global position system (GPS) signals are notreliable. The positioning of the UAV can include determining a positionfor the UAV in a three-dimensional space.

In some embodiments the UAV can weigh less than 1000 g and have agreatest dimension of no more than 100 cm.

The vertices of shapes in the visual patters can be detected and used tocalculate positioning relative to one or more visual markers. The one ormore visual markers can include a ribbon having a visual pattern thatincludes squares. The visual patter can include squares of differentcolors. The verities of the squares can be detected and used tocalculate a dimension of the portion of the visual pattern.

The method can further comprise determining, with aid of one or moreprocessors, a location of the UAV within a three-dimensional coordinatesystem based on the visual patterns. The three-dimensional coordinatesystem can be a global coordinate system. The three-dimensionalcoordinate system can be a local coordinate system.

The method can further comprise identifying the visual patterns andeffecting a response by the UAV based on the identified visual patterns.

In some instances, positioning can comprise calculating a rotation and aposition of the UAV relative to a visual marker using (1) one or moreintrinsic properties of the one or more vision sensors on board the UAV,(2) a known position of the visual marker, and (3) a set of points in animage of the visual marker taken by the vision sensor on board the UAV.

At least one of the one or more vision sensors can be a camera. The UAVcan be conducting surveillance within the environment while movingautonomously. The positioning of the UAV can occur with aid of one ormore processors that are on-board the UAV.

In some embodiments, the method can further comprise detecting, with aidof one or more sensor on-board the UAV, an obstacle; and adjustingdirection of the UAV's flight to avoid the obstacle.

Positioning can be determined with aid of an infrared or radar detector.

In an aspect of the disclosure, a method of positioning an unmannedaerial vehicle (UAV) can comprise: capturing, with aid of one or morevision sensors of the UAV while the UAV is in flight, an image includinga visual marker having a visual pattern and affixed to a surface withinthe environment; calculating a dimension of at least a portion of thevisual pattern of the visual marker in the image; and determining, withaid of one or more processors, a location of the UAV within athree-dimensional coordinate system, based on the dimension of theportion of the visual pattern.

The environment can be an indoor environment. The environment can be anenvironment in which global position system (GPS) signals are notreliable.

In some embodiments the UAV can weigh less than 1000 g and have agreatest dimension of no more than 100 cm.

The dimension can be between vertices of shapes in the visual pattern.

The vertices of shapes in the visual patters can be detected and used tocalculate positioning relative to one or more visual markers. The one ormore visual markers can include a ribbon having a visual pattern thatincludes squares. The visual patter can include squares of differentcolors. The verities of the squares can be detected and used tocalculate a dimension of the portion of the visual pattern.

The method can further comprise determining, with aid of one or moreprocessors, a location of the UAV within a three-dimensional coordinatesystem based on the visual patterns. The three-dimensional coordinatesystem can be a global coordinate system. The three-dimensionalcoordinate system can be a local coordinate system.

The method can further comprise identifying the visual patterns andeffecting a response by the UAV based on the identified visual patterns.

In some instances, positioning can comprise calculating a rotation and aposition of the UAV relative to a visual marker using (1) one or moreintrinsic properties of the one or more vision sensors on board the UAV,(2) a known position of the visual marker, and (3) a set of points in animage of the visual marker taken by the vision sensor on board the UAV.The dimension can be used to calculate a distance of the UAV from thevisual marker. The dimension can be used to calculate and angle of theUAV relative to the visual marker.

At least one of the one or more vision sensors can be a camera. The UAVcan be patrolling the environment autonomously when the one or morevision sensors capture the image. One or more processors can be on-boardthe UAV.

In some embodiments, the method can further comprise detecting, with aidof one or more sensor on-board the UAV, an obstacle; and adjustingdirection of the UAV's flight to avoid the obstacle.

The method can further comprise detecting, with aid of one or moresensor on-board the UAV, an obstacle; and adjusting direction of theUAV's flight to avoid the obstacle.

Positioning can be determined with aid of an infrared or radar detector.

In an aspect of the disclosure, a system positioning an unmanned aerialvehicle (UAV) within an environment can comprise: one or more visionsensors of the UAV configured to capture an image including a visualmarker having a visual pattern and affixed to a surface within theenvironment, while the UAV is in flight; and one or more processors,individually or collectively configured to: (1) calculate a dimension ofat least a portion of the visual pattern of the visual marker in theimage; and (2) determine a location of the UAV within athree-dimensional coordinate system, based on the dimension of theportion of the visual pattern.

The environment can be an indoor environment. The environment can be anenvironment in which GPS signals are not reliable. The UAV can weighless than 1000 g and have a greatest dimension of no more than 100 cm.

The marker dimension can be between vertices of shapes in the visualpattern. The visual marker can be a ribbon having a visual pattern thatincludes squares. The visual pattern can include squares of differentcolors. The vertices of the squares can be detected and used tocalculate the dimension of the portion of the visual patter. Thethree-dimensional coordinate system can be a three-dimensional globalcoordinate system. The three dimensional coordinate system can be athree-dimensional local coordinate system. The visual pattern can beidentified and a response can be effected by the UAV based on theidentified visual pattern.

The one or more processors can be configured to calculate a rotation anda position of the UAV relative to a visual marker using one or moreintrinsic properties of the one or more vision sensors on board the UAV,a known position of the visual marker, and a set of points in an imageof the visual marker taken by the vision sensor on board the UAV.

At least one of the one or more vision sensors can be a camera. Thedimension can be used to calculate a distance of the UAV from the visualmarker. The dimension can be used to calculate an angle of the UAVrelative to the visual marker.

The UAV can be patrolling the environment autonomously when the one ormore vision sensors capture the image. The one or more processors can beon-board the UAV. The UAV can be further configured to detect, with aidof one or more sensor on-board the UAV, an obstacle; and adjust adirection of the UAV's flight to avoid the obstacle.

Positioning can be determined with aid of an infrared or radar detector.

In another aspect of the disclosure, a method of positioning an unmannedaerial vehicle (UAV) within an environment can comprise: capturing, withaid of one or more vision sensors of the UAV while the UAV is in flight,an image including a visual marker having a unique visual pattern withinthe environment and affixed to a surface within the environment;identifying and distinguishing the visual marker from a plurality ofdifferent markers based on the unique visual pattern of the visualmarker in the image, wherein the identified visual marker has a knownlocation that is different from locations of the plurality of differentmarkers; and determining, with aid of one or more processors, a locationof the UAV within a three-dimensional coordinate system, based on theknown location of the identified visual marker.

The environment can be an indoor environment. The environment can be anenvironment in which GPS signals are not reliable. The UAV can weighless than 1000 g and have a greatest dimension of no more than 100 cm.

The vertices of shapes in the visual patterns can be detected and usedto calculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual pattern thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The method can further comprise calculating a dimension of at least aportion of the unique visual pattern of the visual marker in the image;and determining, with aid of the one or more processors, the location ofa UAV within the three-dimensional coordinate system, based on thedimension of the portion of the unique visual pattern.

The three-dimensional coordinate system can be a three-dimensionalglobal coordinate system. The three dimensional coordinate system can bea three-dimensional local coordinate system.

The method can further comprise identifying the visual patterns andeffecting a response by the UAV based on the identified visual patterns.

The method can further comprise comprising positioning the UAV withinthe environment by calculating a rotation and a position of the UAVrelative to a visual marker using one or more intrinsic properties ofthe one or more vision sensors on board the UAV, a known position of thevisual marker, and a set of points in an image of the visual markertaken by the vision sensor on board the UAV.

The one or more processors can be on-board the UAV. The UAV can bepatrolling the environment autonomously when the one or more visionsensors capture the image. The method can further comprise detecting,with aid of one or more sensor on-board the UAV, an obstacle; andadjusting direction of the UAV's flight to avoid the obstacle.

Positioning can be determined with aid of an infrared or radar detector.

In another aspect of the disclosure, a system for positioning anunmanned aerial vehicle (UAV) within an environment can comprise: one ormore vision sensors of the UAV configured to capture an image includinga visual marker having a unique visual pattern within the environmentand affixed to a surface within the environment, while the UAV isflight; and one or more processors, individually or collectivelyconfigured to: identify and distinguish the visual marker from aplurality of different markers based on the unique visual pattern of thevisual marker in the image, wherein the identified visual marker has aknown location that is different from locations of the plurality ofdifferent markers; and determine a location of the UAV within athree-dimensional coordinate system, based on the known location of theidentified visual marker.

The environment can be an indoor environment. The environment can be anenvironment in which GPS signals are not reliable. The UAV can weighless than 1000 g and have a greatest dimension of no more than 1000 cm.

The vertices of shapes in the visual pattern can be detected and used tocalculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual patter thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The one or more processors can be configured to calculate a dimension ofat least a portion of the unique visual pattern of the visual marker inthe image; and determine the location of a UAV within thethree-dimensional coordinate system, based on the dimension of theportion of the unique visual pattern.

The three-dimensional coordinate system can be a three-dimensionalglobal coordinate system. The three dimensional coordinate system can bea three-dimensional local coordinate system.

The visual pattern can be identified and a response can be effected bythe UAV based on the identified visual patterns.

One or more processors can be configured to calculate a rotation and aposition of the UAV relative to a visual marker using one or moreintrinsic properties of the one or more vision sensors on board the UAV,a known position of the visual marker, and a set of points in an imageof the visual marker taken by the vision sensor on board the UAV.

At least one of the one or more vision sensors can be a camera. The UAVcan be patrolling the environment autonomously when the one or morevision sensors capture the image. One or more processors can be on-boardthe UAV. Positioning can be determined with aid of an infrared or radardetector.

In another aspect of the disclosure, a method of positioning an unmannedaerial vehicle (UAV) within an environment can comprise: capturing, withaid of one or more vision sensors of the UAV while the UAV is in flight,an image including a visual marker having a visual pattern; identifyingand distinguishing, with aid of one or more processors, the visualpattern from a plurality of different patterns, wherein the identifiedvisual pattern elicits a response from the UAV that is different fromresponses elicited by the plurality of different patterns; and effectingthe response by the UAV to the identified visual pattern of the visualmarker.

The environment can be an indoor environment. The environment can be anenvironment in which GPS signals are not reliable. The UAV can weighless than 1000 g and have a greatest dimension of no more than 100 cm.

The vertices of shapes in the visual patterns can be detected and usedto calculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual pattern thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The response can be independent of flight of the UAV. The response caninclude adjusting a position of a payload of the UAV. The payload can bea camera on-board the UAV. The response can include capturing an imageof a selected subject. The response can occur within a predeterminedamount of time after the visual patter is identified and distinguished.The response can occur within a predetermined distance after the visualpatter is identified and distinguished. The response can includechanging an attitude of the UAV. The response can include causing theUAV to flying in a particular direction and/or with a particular speed.The response can include causing the UAV to fly up or down stairs. Theresponse can affect flight of the UAV. The response can include causingthe flight of the UAV to follow a set of preset instructions. In somecases, the visual marker is a ribbon and the response includes causingthe flight of the UAV to follow along the ribbon.

The method can further comprise calculating a dimension of at least aportion of the unique visual pattern of the visual marker in the image;and determining, with aid of the one or more processors, the location ofa UAV within the three-dimensional coordinate system, based on thedimension of the portion of the unique visual pattern.

The three-dimensional coordinate system can be a three-dimensionalglobal coordinate system. The three dimensional coordinate system can bea three-dimensional local coordinate system. The local coordinate of theUAV can be calculated by adopting a perspective N points (PNP) algorithm

The method can further comprise comprising positioning the UAV bycalculating a rotation and a position of the UAV relative to a visualmarker using one or more intrinsic properties of the one or more visionsensors on board the UAV, a known position of the visual marker, and aset of points in an image of the visual marker taken by the visionsensor on board the UAV. At least one of the one or more vision sensorscan be a camera.

The one or more processors can be on-board the UAV. The UAV can bepatrolling the environment autonomously when the one or more visionsensors capture the image. The method can further comprise detecting,with aid of one or more sensor on-board the UAV, an obstacle; andadjusting direction of the UAV's flight to avoid the obstacle. Thepositioning can also be determined with aid of an infrared or radardetector.

In another aspect of the disclosure, a system for positioning anunmanned aerial vehicle (UAV) within an environment can comprise: one ormore vision sensors of the UAV configured to capture an image includinga visual marker having a visual pattern, while the UAV is flight; andone or more processors, individually or collectively configured to: (1)identify and distinguish, with aid of one or more processors, the visualpattern from a plurality of different patterns, wherein the identifiedvisual pattern elicits a response from the UAV that is different fromresponses elicited by the plurality of different patterns; and (2)effect the response by the UAV to the identified visual pattern of thevisual marker.

The environment can be an indoor environment. The environment can be anenvironment in which GPS signals are not reliable. The UAV can weighless than 1000 g and have a greatest dimension of no more than 100 cm.

The vertices of shapes in the visual patterns can be detected and usedto calculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual pattern thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The response can be independent of flight of the UAV. The response caninclude adjusting a position of a payload of the UAV. The payload can bea camera on-board the UAV. The response can include capturing an imageof a selected subject. The response can occur within a predeterminedamount of time after the visual patter is identified and distinguished.The response can occur within a predetermined distance after the visualpatter is identified and distinguished. The response can includechanging an attitude of the UAV. The response can include causing theUAV to flying in a particular direction and/or with a particular speed.The response can include causing the UAV to fly up or down stairs. Theresponse can affect flight of the UAV. The response can include causingthe flight of the UAV to follow a set of preset instructions. In somecases, the visual marker is a ribbon and the response includes causingthe flight of the UAV to follow along the ribbon.

The system can further comprise calculating a dimension of at least aportion of the visual pattern of the visual marker in the image; anddetermining, with aid of the one or more processors, the location of aUAV within the three-dimensional coordinate system, based on thedimension of the portion of the visual pattern.

The three-dimensional coordinate system can be a three-dimensionalglobal coordinate system. The three dimensional coordinate system can bea three-dimensional local coordinate system. The local coordinate of theUAV can be calculated by adopting a perspective N points (PNP) algorithm

The one or more processors are configured to calculate a rotation and aposition of the UAV relative to a visual marker using one or moreintrinsic properties of the one or more vision sensors on board the UAV,a known position of the visual marker, and a set of points in an imageof the visual marker taken by the vision sensor on board the UAV. Atleast one of the one or more vision sensors can be a camera.

The UAV can be patrolling the environment autonomously when the one ormore vision sensors capture the image. The one or more processors can beon-board the UAV. The system can further comprise detecting, with aid ofone or more sensor on-board the UAV, an obstacle; and adjustingdirection of the UAV's flight to avoid the obstacle. The positioning canalso be determined with aid of an infrared or radar detector.

In another aspect of the disclosure, a method of positioning an unmannedaerial vehicle (UAV) within an environment can comprise: providing aplurality of visual markers within the environment, wherein each of thevisual markers within the environment have a visual pattern that isunique for the environment; capturing, with aid of one or more visionsensors of the UAV while the UAV is in flight within the environment, animage including a detected visual marker having an identified visualpattern from said plurality of visual markers; effecting, with aid ofone or more processors, a navigation response by the UAV to the detectedvisual marker having the identified visual pattern.

The navigation response can include determining a location of the UAV ina three-dimensional coordinate system. The navigation response caninclude causing the UAV to fly in accordance with a set of preset flightcommands. The navigation response can include causing the UAV to fly toa predetermined location relative to the detected visual marker. Thenavigation response can include causing the UAV to fly to a locationfrom which the one or more vision sensors of the UAV are configured tocapture an image of a different visual marker from the detected visualmarker. The environment can be an indoor environment. The environmentcan be an environment in which global position systems (GPS) signals arenot reliable. The UAV can weigh less than 1000 g and have a greatestdimension of no more than 100 cm.

The vertices of shapes in the visual patterns can be detected and usedto calculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual pattern thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The response can be independent of flight of the UAV. The response caninclude adjusting a position of a payload of the UAV. The payload can bea camera on-board the UAV. The response can include capturing an imageof a selected subject. The response can occur within a predeterminedamount of time after the visual patter is identified and distinguished.The response can occur within a predetermined distance after the visualpatter is identified and distinguished. The response can includechanging an attitude of the UAV. The response can include causing theUAV to flying in a particular direction and/or with a particular speed.The response can include causing the UAV to fly up or down stairs. Theresponse can affect flight of the UAV. The response can include causingthe flight of the UAV to follow a set of preset instructions. In somecases, the visual marker is a ribbon and the response includes causingthe flight of the UAV to follow along the ribbon.

The method can further comprise calculating a dimension of at least aportion of the visual pattern of the visual marker in the image; anddetermining, with aid of the one or more processors, the location of aUAV within the three-dimensional coordinate system, based on thedimension of the portion of the visual pattern.

The three-dimensional coordinate system can be a three-dimensionalglobal coordinate system. The three dimensional coordinate system can bea three-dimensional local coordinate system. The local coordinate of theUAV can be calculated by adopting a perspective N points (PNP) algorithm

The method can further comprise positioning the UAV within theenvironment by calculating a rotation and a position of the UAV relativeto a visual marker using one or more intrinsic properties of the one ormore vision sensors on board the UAV, a known position of the visualmarker, and a set of points in an image of the visual marker taken bythe vision sensor on board the UAV.

At least one of the one or more vision sensors can be a camera. The UAVcan be patrolling the environment autonomously when the one or morevision sensors capture the image. The one or more processors can beon-board the UAV. The method can further comprise detecting, with aid ofone or more sensor on-board the UAV, an obstacle; and adjustingdirection of the UAV's flight to avoid the obstacle. The positioning canalso be determined with aid of an infrared or radar detector.

In another aspect of the disclosure, method of positioning an unmannedaerial vehicle (UAV) within an environment can comprise: providing acontinuous visual marker within the environment, wherein the continuousvisual marker provides a path within the environment; capturing, withaid of one or more vision sensors of the UAV while the UAV is in flightwithin the environment, an image including at least a portion of thecontinuous visual marker having a visual pattern; effecting a flightresponse by the UAV to keep the continuous visual marker within visualrange of the UAV.

The continuous marker can be a ribbon. The environment can be an indoorenvironment. The environment can be an environment in which GPS signalsare not reliable. The UAV can weigh less than 1000 g and have a greatestdimension of no more than 100 cm.

The vertices of shapes in the visual patterns can be detected and usedto calculate positioning relative to the one or more visual markers. Thevisual marker can be a ribbon having a unique visual pattern thatincludes squares. The unique visual patter can include squares ofdifferent colors. The unique visual pattern can include AprilTags. Theunique visual pattern can include QR codes. The unique visual patterncan include barcodes.

The response can be independent of flight of the UAV. The response caninclude adjusting a position of a payload of the UAV. The payload can bea camera on-board the UAV. The response can include capturing an imageof a selected subject. The response can occur within a predeterminedamount of time after the visual patter is identified and distinguished.The response can occur within a predetermined distance after the visualpatter is identified and distinguished. The response can includechanging an attitude of the UAV. The response can include causing theUAV to flying in a particular direction and/or with a particular speed.The response can include causing the UAV to fly up or down stairs. Theresponse can affect flight of the UAV. The response can include causingthe flight of the UAV to follow a set of preset instructions. In somecases, the visual marker is a ribbon and the response includes causingthe flight of the UAV to follow along the ribbon.

The UAV can be patrolling the environment autonomously when the one ormore vision sensors capture the image. The method can further comprisedetecting, with aid of one or more sensor on-board the UAV, an obstacle;and adjusting direction of the UAV's flight to avoid the obstacle.Positioning can also be determined with aid of an infrared or radardetector.

Other objects and features of the present disclosure will becomeapparent by a review of the specification, claims, and appended figures.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present disclosure will be obtained by reference tothe following detailed description that sets forth illustrativeembodiments, in which the principles of the disclosure are utilized, andthe accompanying drawings of which:

FIG. 1 shows a UAV configured to conduct surveillance in an environment.

FIG. 2 shows an example of a visual marker.

FIG. 3 shows an example of vertices of a visual marker that may bedetected by a UAV.

FIG. 4 shows a UAV with on board components.

FIG. 5 shows a UAV detecting a visual marker from two verticalpositions.

FIG. 6 shows a UAV detecting a visual marker from two horizontalpositions.

FIG. 7 shows a UAV detecting and avoiding an obstacle by determining aposition based on detection of a visual marker.

FIG. 8 shows a UAV eliminating false detection events of visual markers.

FIG. 9 shows a visual marker including a marking to communicate andinstruction to a UAV.

FIG. 10 shows a UAV following a ribbon of visual markers along a path.

FIG. 11 shows a UAV responding to position instructions from a pluralityof visual markers.

FIG. 12 shows a UAV interpreting an environment from information encodedon visual markers.

FIG. 13 shows a UAV adjusting the position of a payload in response toan instruction encoded by a visual marker.

FIG. 14 illustrates an unmanned aerial vehicle, in accordance with anembodiment of the disclosure.

FIG. 15 illustrates a movable object including a carrier and a payload,in accordance with an embodiment of the disclosure.

FIG. 16 is a schematic illustration by way of block diagram of a systemfor controlling a movable object, in accordance with an embodiment ofthe disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The systems, devices, and methods of the present disclosure providepositioning methods for automatically aiding an unmanned aerial vehicle(UAV) in navigating an environment. Description of the UAV may beapplied to any other type of unmanned vehicle, or any other type ofmovable object.

A UAV may be provided to patrol an environment to collect informationabout the environment or one or more subjects in the environment. TheUAV can follow a defined path through an environment or the UAV canfollow an unstructured path. The UAV can autonomously orsemi-autonomously follow a subject of interested and collect sensorydata pertaining to the subject on interest. Sensory data can be visualdata, audio data, location data, and/or movement data. A UAV in asurveillance environment can receive commands from an off board controlcenter through a communication network. The communication network can bewired or wireless. In some cases the UAV can receive communication froma pattern recognition mechanism that interprets instructions encoded ona visual marker placed in an environment.

The UAV can collect visual information in an environment. Visualinformation can be collected using a vision sensor. In some cases, avision sensor can be a camera. A vision sensor can also be used todetect a visual marker to determine the location of a UAV in anenvironment. In some cases, one vision sensor can be used for bothdetection visual markers and collecting visual information in anenvironment. In other cases, the UAV can have at least two visionsensors such that a first vision sensor is configured to detect visualmarkers and a second vision sensor is configured to collect visualinformation in the environment. The vision sensors can be located in abody of the UAV or the vision sensors can be an external payloadattached to the UAV. The UAV can also collect audio information in anenvironment using an audio sensor, for example a microphone. Thecollected information can be collected and stored on a memory storagedevice located on board or off board the UAV. The collected informationcan be monitored by a user in real time. Alternatively, the collectedinformation can be stored on a memory storage device and reviewed by auser after collection. In some cases the UAV can emit an audio or visualsignal as a warning or indication that the UAV has entered or exited aspecified location.

Visual markers that may be detected by one or more vision sensors of theUAV may be distributed about an environment. The visual markers may ormay not be visually distinguishable from one another. The visual markersmay be at known locations, and may be useful for determining a UAVlocation. The visual markers may also be useful for aiding a UAV innavigating an environment and/or performing a task within theenvironment.

The UAV can be configured to perform surveillance tasks in anenvironment to carry out a security assessment. The UAV can collectinformation in the environment using a plurality of sensors incommunication with one or more processors on board or off board the UAV.The processors can interpret the collected information in order todetermine events that may indicate a hazard or unsafe condition. The UAVcan be instructed by the processor to produce and alarm when a hazard orunsafe condition is detected by the processor.

FIG. 1 shows an example of an unmanned aerial vehicle (UAV) 101. The UAV101 can have one or more sensors. The UAV 101 may comprise one or morevision sensors 102 such as an image sensor. For example, an image sensormay be a monocular camera, stereo vision camera, radar, sonar, or aninfrared camera. The UAV 101 may further comprise other sensors that maybe used to determine a location of the UAV, such as global positioningsystem (GPS) sensors, inertial sensors which may be used as part of orseparately from an inertial measurement unit (IMU) (e.g.,accelerometers, gyroscopes, magnetometers), lidar, ultrasonic sensors,acoustic sensors, WiFi sensors. The UAV can have sensor on board onboard the UAV that collect information directly from an environmentwithout contacting an additional component off board the UAV foradditional information or processing. For example, a sensor thatcollects data directly in an environment can be a vision or audiosensor. Alternatively, the UAV can have sensors that are on board theUAV but contact one or more components off board the UAV to collect dataabout an environment. For example, a sensor that contacts a componentoff board the UAV to collect data about an environment may be a GPSsensor or another sensor that relies on connection to another device,such as a satellite, tower, router, server, or other external device.Various examples of sensors may include, but are not limited to,location sensors (e.g., global positioning system (GPS) sensors, mobiledevice transmitters enabling location triangulation), vision sensors(e.g., imaging devices capable of detecting visible, infrared, orultraviolet light, such as cameras), proximity or range sensors (e.g.,ultrasonic sensors, lidar, time-of-flight or depth cameras), inertialsensors (e.g., accelerometers, gyroscopes, inertial measurement units(IMUs)), altitude sensors, attitude sensors (e.g., compasses) pressuresensors (e.g., barometers), audio sensors (e.g., microphones) or fieldsensors (e.g., magnetometers, electromagnetic sensors). Any suitablenumber and combination of sensors can be used, such as one, two, three,four, five, or more sensors. Optionally, the data can be received fromsensors of different types (e.g., two, three, four, five, or moretypes). Sensors of different types may measure different types ofsignals or information (e.g., position, orientation, velocity,acceleration, proximity, pressure, etc.) and/or utilize different typesof measurement techniques to obtain data. For instance, the sensors mayinclude any suitable combination of active sensors (e.g., sensors thatgenerate and measure energy from their own energy source) and passivesensors (e.g., sensors that detect available energy). As anotherexample, some sensors may generate absolute measurement data that isprovided in terms of a global coordinate system (e.g., position dataprovided by a GPS sensor, attitude data provided by a compass ormagnetometer), while other sensors may generate relative measurementdata that is provided in terms of a local coordinate system (e.g.,relative angular velocity provided by a gyroscope; relativetranslational acceleration provided by an accelerometer; relativeattitude information provided by a vision sensor; relative distanceinformation provided by an ultrasonic sensor, lidar, or time-of-flightcamera). The sensors onboard or off board the UAV may collectinformation such as location of the UAV, location of other objects,orientation of the UAV 101, or environmental information. A singlesensor may be able to collect a complete set of information in anenvironment or a group of sensors may work together to collect acomplete set of information in an environment. Sensors may be used formapping of a location, navigation between locations, detection ofobstacles, or detection of a target. Sensors may be used forsurveillance of an environment or a subject of interest 103.

Any description herein of a UAV 101 may apply to any type of movableobject. The description of a UAV may apply to any type of unmannedmovable object (e.g., which may traverse the air, land, water, orspace). The UAV may be capable of responding to commands from a remotecontroller. The remote controller may be not connected to the UAV, theremote controller may communicate with the UAV wirelessly from adistance. In some instances, the UAV may be capable of operatingautonomously or semi-autonomously. The UAV may be capable of following aset of pre-programmed instructions. In some instances, the UAV mayoperate semi-autonomously by responding to one or more commands from aremote controller while otherwise operating autonomously. For instance,one or more commands from a remote controller may initiate a sequence ofautonomous or semi-autonomous actions by the UAV in accordance with oneor more parameters.

The UAV 101 may be an aerial vehicle. The UAV may have one or morepropulsion units that may permit the UAV to move about in the air. Theone or more propulsion units may enable the UAV to move about one ormore, two or more, three or more, four or more, five or more, six ormore degrees of freedom. In some instances, the UAV may be able torotate about one, two, three or more axes of rotation. The axes ofrotation may be orthogonal to one another. The axes of rotation mayremain orthogonal to one another throughout the course of the UAV'sflight. The axes of rotation may include a pitch axis, roll axis, and/oryaw axis. The UAV may be able to move along one or more dimensions. Forexample, the UAV may be able to move upwards due to the lift generatedby one or more rotors. In some instances, the UAV may be capable ofmoving along a Z axis (which may be up relative to the UAV orientation),an X axis, and/or a Y axis (which may be lateral). The UAV may becapable of moving along one, two, or three axes that may be orthogonalto one another.

The UAV 101 may be a rotorcraft. In some instances, the UAV may be amulti-rotor craft that may include a plurality of rotors. The pluralityof rotors may be capable of rotating to generate lift for the UAV. Therotors may be propulsion units that may enable the UAV to move aboutfreely through the air. The rotors may rotate at the same rate and/ormay generate the same amount of lift or thrust. The rotors mayoptionally rotate at varying rates, which may generate different amountsof lift or thrust and/or permit the UAV to rotate. In some instances,one, two, three, four, five, six, seven, eight, nine, ten, or morerotors may be provided on a UAV. The rotors may be arranged so thattheir axes of rotation are parallel to one another. In some instances,the rotors may have axes of rotation that are at any angle relative toone another, which may affect the motion of the UAV.

The UAV shown may have a plurality of rotors. The rotors may connect tothe body of the UAV which may comprise a control unit, one or moresensors, processor, and a power source. The sensors may include visionsensors and/or other sensors that may collect information about the UAVenvironment. The information from the sensors may be used to determine alocation of the UAV. The rotors may be connected to the body via one ormore arms or extensions that may branch from a central portion of thebody. For example, one or more arms may extend radially from a centralbody of the UAV, and may have rotors at or near the ends of the arms.

A vertical position and/or velocity of the UAV may be controlled bymaintaining and/or adjusting output to one or more propulsion units ofthe UAV. For example, increasing the speed of rotation of one or morerotors of the UAV may aid in causing the UAV to increase in altitude orincrease in altitude at a faster rate. Increasing the speed of rotationof the one or more rotors may increase the thrust of the rotors.Decreasing the speed of rotation of one or more rotors of the UAV mayaid in causing the UAV to decrease in altitude or decrease in altitudeat a faster rate. Decreasing the speed of rotation of the one or morerotors may decrease the thrust of the one or more rotors. When a UAV istaking off, the output may be provided to the propulsion units may beincreased from its previous landed state. When the UAV is landing, theoutput provided to the propulsion units may be decreased from itsprevious flight state. The UAV may be configured to take off and/or landin a substantially vertical manner.

A lateral position and/or velocity of the UAV may be controlled bymaintaining and/or adjusting output to one or more propulsion units ofthe UAV. The altitude of the UAV and the speed of rotation of one ormore rotors of the UAV may affect the lateral movement of the UAV. Forexample, the UAV may be tilted in a particular direction to move in thatdirection and the speed of the rotors of the UAV may affect the speed ofthe lateral movement and/or trajectory of movement. Lateral positionand/or velocity of the UAV may be controlled by varying or maintainingthe speed of rotation of one or more rotors of the UAV.

The UAV 101 may be of small dimensions. The UAV may be capable of beinglifted and/or carried by a human. The UAV may be capable of beingcarried by a human in one hand.

The UAV 101 may have a greatest dimension (e.g., length, width, height,diagonal, diameter) of no more than 100 cm. In some instances, thegreatest dimension may be less than or equal to 1 mm, 5 mm, 1 cm, 3 cm,5 cm, 10 cm, 12 cm, 15 cm, 20 cm, 25 cm, 30 cm, 35 cm, 40 cm, 45 cm, 50cm, 55 cm, 60 cm, 65 cm, 70 cm, 75 cm, 80 cm, 85 cm, 90 cm, 95 cm, 100cm, 110 cm, 120 cm, 130 cm, 140 cm, 150 cm, 160 cm, 170 cm, 180 cm, 190cm, 200 cm, 220 cm, 250 cm, or 300 cm. Optionally, the greatestdimension of the UAV may be greater than or equal to any of the valuesdescribed herein. The UAV may have a greatest dimension falling within arange between any two of the values described herein.

The UAV 101 may be lightweight. For example, the UAV may weigh less thanor equal to 1 mg, 5 mg, 10 mg, 50 mg, 100 mg, 500 mg, 1 g, 2 g, 3 g, 5g, 7 g, 10 g, 12 g, 15 g, 20 g, 25 g, 30 g, 35 g, 40 g, 45 g, 50 g, 60g, 70 g, 80 g, 90 g, 100 g, 120 g, 150 g, 200 g, 250 g, 300 g, 350 g,400 g, 450 g, 500 g, 600 g, 700 g, 800 g, 900 g, 1 kg, 1.1 kg, 1.2 kg,1.3 kg, 1.4 kg, 1.5 kg, 1.7 kg, 2 kg, 2.2 kg, 2.5 kg, 3 kg, 3.5 kg, 4kg, 4.5 kg, 5 kg, 5.5 kg, 6 kg, 6.5 kg, 7 kg, 7.5 kg, 8 kg, 8.5 kg, 9kg, 9.5 kg, 10 kg, 11 kg, 12 kg, 13 kg, 14 kg, 15 kg, 17 kg, or 20 kg.The UAV may have a weight greater than or equal to any of the valuesdescribed herein. The UAV may have a weight falling within a rangebetween any two of the values described herein.

The UAV 101 may comprise vision sensors, such as a monocular camera,stereo vision camera, radar, sonar, or an infrared camera. The UAV 101may further comprise sensors, such as GPS, IMU, lidar, or any othertypes of sensors described elsewhere herein. The sensors onboard the UAVmay collect information such as location of the UAV, location of otherobjects, orientation of the UAV 101, or environmental information. Asingle sensor may be able to fully determine any one of theaforementioned parameters or a group of sensors may work together todetermine one of the listed parameters. Sensors may be used for mappingof a location, navigation between locations, detection of obstacles,detection of a target, or surveillance of an object or environment ofinterest.

The sensors may be located onboard or off board the UAV. The onboardsensors may be located on the body of the UAV 101. The sensors may beattached to the outside of the body of the UAV 101 and/or the sensor maybe attached to the inside of the body of the UAV 101. The sensors may becentrally located in a single region on the body. Alternatively, thesensors may be located in different locations on the body. The sensorsmay be permanently or removably attached to the UAV 101. The UAV 101 mayhave a carrier which may be configured to carry a payload. Sensors maybe attached to the carrier.

The sensors may be characterized by one or more sensors parameters. Thesensor parameters may be intrinsic or extrinsic parameters. An intrinsicparameter may relate to the internal configuration of a sensor. Exampleof intrinsic parameters may include focal length, scale factor, radialdistortion coefficients, and tangential distortion coefficients.Intrinsic parameters may be any parameters that are dependent onhardware configurations, in some cases the intrinsic parameters may beset by a factory setting for the sensor. Extrinsic parameters may relateto the spatial relationship between any two or more sensors. Each sensormay have a relative coordinate system independent of other sensors onboard the UAV. Extrinsic properties may be important for sensor fusion,combining data from sensors in different locations on the UAV. Sensorfusion may involve a process of transforming the relative coordinates ofa given sensor to match the reference frame of another sensor.

A UAV may be configured to conduct surveillance in a space orenvironment. The space may be an outdoor or indoor space or acombination of indoor and outdoor spaces. The UAV may automatically orsemi-automatically position itself in the space. The UAV may positionitself with a location sensor, for example a GPS sensor. Alternatively,the UAV may position itself by interpreting images of visual markers.Visual markers may be an alternative locating mechanism to locatingsensors in environments where locating sensors are not reliable. Visualmarkers may be used in combination with one or more types of locatingsensors. The visual markers may have a locating marker. The visualmarkers may be adhered or attached to one or more surfaces in the spaceor environment.

Data regarding flight of the UAV may be correlated with data captured byone or more sensor of the UAV used for surveillance. For example alocation of the UAV (e.g., latitude, longitude, altitude or any othercoordinate system), attitude of the UAV (e.g., angle about a roll,pitch, and/or yaw axis) may be stored. The information about the UAV,such as location and attitude may be correlated with timing data. Forexample, a clock may be provided on-board the UAV or an external devicethat may keep track of time for positioning of the UAV. The time datamay be associated with the corresponding location and/or attitude of theUAV. This time-based data may be correlated with data captured by one ormore sensors of the UAV, such as image data captured by a cameraon-board the UAV. Optionally, camera data (e.g., attitude of camerarelative to the UAV and/or timing information) may be correlated withthe other data. In some embodiments, the UAV may be able to rotatefreely about at least 45 degrees, 90 degrees, 180 degrees, 270 degrees,360 degrees, or 720 degrees about any of the axes described herein. Apayload (e.g., camera) may be able to rotate freely about at least 45degrees, 90 degrees, 180 degrees, 270 degrees, 360 degrees, or 720degrees about any of the axes described herein relative to the UAV. Thisdata may be useful for determining positioning of the UAV andunderstanding the positioning of a region captured by the sensor (e.g.,field of view captured by the camera) during surveillance.

FIG. 2 shows an example of a locating marker. The locating marker can bepresent in an indoor environment, such as within a building. Theinterior of the building may have weak or absent GPS signal. Thelocating marker may be adhered to a surface in an environment. One ormore locating markers and/or a continuous ribbon of locating markers canbe present throughout an environment. One or more locating markersand/or a continuous ribbon of locating markers can be adhered to asurface or surfaces in a room, hallway, alcove, closet, or otherenclosure. The environment may be an indoor environment. The locatingmarker may have a discrete shape. For instance, the locating marker maybe a square, rectangle, circle, triangle, hexagon, or have any othershape. The locating marker can be adhered to a ceiling, wall, supportbeam, object, or floor in an indoor environment. The locating marker canbe a continuous ribbon. The locating marker can be a series ofunconnected markers of uniform or non-uniform dimension. The locatingmarker may be black and white or the locating marker may be a pluralityof colors. The locating marker may comprise a plurality of geometricshapes with recognizable vertices. The plurality of geometric shapes maycomprise a visual pattern. The geometric shapes may be triangles,squares, pentagons, hexagons, or any other regular or irregular polygon.The locating marker visual pattern may comprise at least one type ofpolygon. The locating marker can comprise a repeating pattern,checkerboard, rows and/or columns of repeating polygons, or an array ofpolygons. The locating marker can have shapes of alternating colors in aconsistent repetitive pattern. In the example shown in FIG. 2 the markercomprises a set of squares. The example maker in FIG. 2 shows black andwhite squares, alternatively the squares may be other colors.

In an alternate embodiment, the marker may not be a visual marker. Themarker can be detected by another sensory interaction. In some cases,the marker can be an audio marker. The marker can provide an audio orvisual warning when the UAV leaves or enters a specified location. Themarker can emit an acoustic disturbance in a frequency range that can bedetected by the UAV. The frequency range of the emitted acousticdisturbance can be in a frequency range outside of a frequency rangedetected by humans. The UAV can determine its location by analyzing adistortion in the acoustic disturbance. The acoustic disturbance caninstruct a UAV to move to perform a mission. In another case, the markercan emit an RFID, radio, or WiFi signal that can be detected by the UAVand processed by the UAV to determine position. In some embodiments themarker can emit a detectable substance, for example a gas. Thedetectable substance can be safe for inhalation and consumption byhumans and other animals. The UAV can determine its location relative tothe marker using the concentration of the gas at a location where theUAV detects the gas and the diffusive properties of the gas.

Different markers at different locations may be distinguishable from oneanother. For example, different markers may emit different signals. Inone example, a sound emitted by a first audio marker may be differentfrom a sound emitted by a second audio marker. The sounds may havedifferent frequencies, harmonics, or patterns. Thus, by recognizing thesound pattern, the UAV may be able to distinguish whether the UAV isnear a first audio marker at a first location vs. a second audio markerat a second location. Similarly, different radios may emit differentsignals, or different gas markers may emit gases of differentcompositions. A UAV may be able to determine its location based on theidentified marker and a known location of the identified marker.

Any description herein of a visual marker or locating marker may applyto any other type of marker, including non-visual markers, and viceversa.

The locating marker can be a manufactured from a paper or plasticmaterial. The locating marker can be a screen display (e.g. liquidcrystals display (LCD), touch screen, LED screen, OLED screen, or plasmascreen display). Alternatively, the locating marker can be projected onto a surface from a projector installed in the environment. The locatingmarker can be a screen display or projector in communication with acomputer unit. The locating marker can be static (e.g. constant) ordynamic (e.g. changeable). The computer unit can change the pattern onthe locating marker. Changing the pattern can change the interaction ofthe UAV with the locating marker. The locating marker can be adhered toa surface with glue, resin, magnets, or hardware components (e.g.screws, fasteners, Velcro, rivets, nails, or snaps). The locating markercan be permanently or removably attached to a surface.

In an environment the visual markers may be uniform. An environment maycontain only one type of visual marker, wherein a type of visual markermay refer to a specific pattern on a visual marker. Differences in thepattern on a visual marker may be differences in the type of shapes inthe pattern, number of shapes in the pattern, color of shapes in thepattern, ratio of sizes of the shapes in the pattern, and/or differencesin a coded pattern. In some cases an environment may contain more thanone type of visual marker. Different visual markers in an environmentmay be recognized by different UAVs. A processor on or off board the UAVmay be configured to distinguish between different visual markers. In anexample different UAVs may be UAVs with different missions, withdifferent functions, or belonging to different companies. In someembodiments a UAV may be configured to recognize and follow only onetype of visual markers.

For example, a first UAV may be configured to recognize a visual markerof a first type without recognizing a visual marker of a second type. Asecond UAV may be configured to recognize a visual marker of a secondtype without recognizing a visual marker of a first type. For example, afirst visual marker type may include alternating black and whitesquares. A second visual marker type may include alternating red andgreen circles. The first UAV may only recognize the black and whitesquare pattern as a visual marker while ignoring the red and greencircle pattern. Similarly, the second UAV may only recognize the red andgreen circle pattern as a visual marker while ignoring the black andwhite square pattern.

The UAV may recognize static patterns. For example, a UAV may recognizea still image as a visual marker. The still image may change from afirst still image to a second still image. For instance, the visualmarker may be implemented as a screen that may show the first stillimage, and then change to a second still image. In some instances, a UAVmay recognize both still images as markers, or different UAVs mayrecognize the still images as markers (e.g., a first UAV may recognize afirst still image as a marker while a second UAV may recognize a secondstill image as a marker). In some instances, a UAV may recognize avisual marker that may change. The actual variance or sequence of changemay be part of what permits the UAV to recognize the visual marker. TheUAV may recognize the change of the visual marker over time in order toidentify the visual marker. For instance, the visual marker may beimplemented as a screen that may show a video sequence.

Visual markers may be interpreted by a UAV to determine the position orlocation of the UAV in an environment. Visual markers can be used toposition a UAV in an environment in cases where global positioningsystem (GPS) signals are not reliable. In an example, GPS signals may beunreliable in indoor environments and in remote locations. The UAV maybe able to determine its location within the environment locally orglobally. A local location may refer to a location relative to anobject, landmark, or known position in an environment. In an example, alocal location can be a cardinal direction extremity of an environment(e.g. north end of a building) or a distance above and angle away froman object or landmark (e.g. at an azimuthal angle of 30°, attitude of45°, and a radial distance of 10 feet from a landmark). Alternatively, aUAV can determine its global location in an environment. Global locationcan be a location in space for example a longitude and latitude. A UAVmay be in an environment to conduct surveillance. Additionally, visualmarkers may communicate an instruction to the UAV. An instruction may bea command related to the UAV position, the UAV mission, the UAV sensors,information processing, and/or interaction with persons, objects, orvehicles in the environment. A UAV may perform the command within aspecified time interval after receiving the command. A UAV may performthe command after traveling a specified distance from the marker thatcommunicated the command. A command may involve controlling the motionof the UAV. In an example, an instruction can be to increase or decreasealtitude, change directional heading, land on a surface, turn on or offa sensor, capture data with a sensor, transmit sensor data to an offboard server or processor, or interact with another vehicle or object inthe environment. An instruction can elicit a response from a UAV. In anexample, a response can be adjusting position of a payload, swapping ofa payload, capturing an image of a selected subject, causing the UAV tofly or head in a specific direction, at a specific speed oracceleration, and/or altitude, or causing the UAV to fly up or downstairs. The instruction to switch a payload can comprise instructions topick up a payload, unload a payload, and/or add an additional payload.The instruction can designate a specific location to bring a payload. Insome cases the instruction can be to sound an alarm provided on boardthe UAV.

The visual marker can be detected by a vision sensor on board a UAV.FIG. 4 shows a diagram with possible components that may exist on boardthe UAV 400. The UAV 400 can have one or more sensors 401. The one ormore sensors can include a vision sensor such as a monocular camera. Theone or more sensors can be in electronic or network communicationthrough a wired or wireless connection with a processor 402. Theprocessor can be configured to conduct an analysis on the sensor data.The analysis can include determining or calculating the position of theUAV using the sensor data. The UAV can further comprise an on boardmemory storage device 403. The memory storage device can be inelectronic or network communication with the processor and/or one ormore sensors through a wired or wireless connection. The memory storagedevice can have stored information about an environment. The storedinformation about the environment can pertain to the geographic locationof the environment in a global coordinate system, the layout of theenvironment, the dimension of one or more spaces in the environment, thefeatures in an environment, and/or locations and dimensions of objectsin the environment. In an example, the stored information can be ablueprint, a map, or a floor plan. The stored information may alsoinclude location of the visual markers. Location of the markers may berelated to the blueprint, map, or floor plan. The information stored onthe memory storage device can be updated periodically or updated when achange occurs in an environment. The information stored on the memorystorage device can be updated by a device in communication with thememory storage device wirelessly or through a wired connection. In analternate embodiment the UAV can be provided with information about anenvironment from a storage device off board the UAV. The UAV 400 canalso include a power source 404, for example a battery, on board theUAV. The UAV can be configured to carry a payload 405, the payload canbe an additional sensor, cargo, or a backup battery. The payload 405 canbe rotated and translated to achieve a plurality of orientationsrelative to the UAV 400.

The visual sensors can communicate an instruction to the UAV through apattern recognition mechanism. The pattern recognition can be performedby the processor on the UAV. The processor can be configured torecognize a pattern and may determine that the pattern communicates acommand for a function or action. In some cases the pattern may be a QRcode, AprilTag, or bar code. The pattern can include any shape, image,icon, letter, symbol, number, or pattern. The pattern may be aone-dimensional pattern, two-dimensional pattern or three-dimensionalpattern. For instance, the pattern may include ridges or bumps that mayprotrude and be recognizable by the UAV.

The UAV may know the location of each visual marker or a ribbon ofvisual markers. The location of the visual markers can be stored in amemory device on board or off board the UAV. Each visual marker orribbon of visual markers can have a unique location. The UAV may beconfigured to locate a specific visual marker and to know the locationof that visual marker.

The UAV can be configured to locate the vertices of the polygon orpolygons on the locating marker. FIG. 3 shows the vertices that can bedetected by a UAV. The UAV can detect the vertices of the squares, orother polygons, and calculate a dimension of the visual pattern. Adimension of the visual pattern can be interpreted to determine alocation of the UAV. The location of the UAV can be determined in athree-dimensional coordinate system. The three-dimensional coordinatesystem can be a global coordinate system or a relative or localcoordinate system.

The UAV can detect a visual pattern and capture an image of the visualpattern with a camera. The UAV may capture an image of a portion of thevisual pattern or the entirety of the visual pattern. The UAV may haveone or more processors configured to process the image of the visualpattern. The one or more processors may be on board or off board theUAV. The processor may be in communication with one or more memorystorage devices. The image of the visual pattern may be stored on thememory storage device. A UAV may determine the distance between two ormore vertices using the one or more processors. A UAV may determine theangle between one or more vertices using the one or more processors. Thedistance and angle between the vertices may comprise a dimension of thevisual marker. The dimension of the visual marker may be used as aninput to calculate the UAV's distance from the marker and angle relativeto the marker.

The dimensional perception of the visual marker by the UAV may be afunction of the location of the UAV relative to the visual marker. Theperceived size of the visual marker may vary with the distance betweenthe UAV and the visual marker. The size of the marker can be describedas the distance between detected vertices of the polygons on the marker.A UAV can determine its height or vertical distance from a marker basedon the apparent size of the marker relative to a known marker size. Themarker can appear smaller than its actual dimension when the UAV is arelatively large vertical distance from the marker. An example of achange in visual perception of the marker as a function of UAV verticalposition (height) is shown in FIG. 5. In a first position the UAV isvertically relatively far from the visual marker 501. The UAV canapproach the visual marker vertically such that in a second position 502the visual marker appears relatively larger. The change in apparentmarker size as a function of vertical distance from the marker can beknown and may be stored in a memory storage device on board or off boardthe UAV. The change in apparent marker size as a function of verticaldistance from the marker can be determined from a calibration or from aknown mathematical relationship.

The UAV can detect a distortion in the shape of the marker as a functionof offset from the marker or horizontal displacement from the marker.Distortion can be measured as a discrepancy in the angles betweencoordinates of the detected vertices of the marker polygons. In a casein which the polygons are squares the UAV may expect the square to havefour vertices at 90° angles to each other. When the UAV is offset from amarker containing squares, or other polygons, the angles between thevertices can appear distorted. In a case in which the UAV is offset froma marker containing squares the shape of the square can be distortedsuch that it appears to the UAV as a rhombus. Similar distortions canoccur with other shapes. An example of the described distortiondetection is described in FIG. 6, a UAV can approach a markerhorizontally. The angle of the vertices can vary as a function ofhorizontal distance from the marker. The angle between the vertices canbe distorted when the UAV is some distance from the marker. In a firstcase 601, when the UAV is a horizontal distance away from the marker theangle between the vertices can appear shortened to the UAV. As the UAVapproaches the marker horizontally, such that the UAV is directly abovethe marker, the angle between the vertices can elongate to the trueundistorted angle 602.

The perceived distortion of the visual marker can be analyzed todetermine the horizontal and vertical location of the UAV relative tothe marker. One or more processors can determine a location or positionof the UAV in a three-dimensional coordinate system based on the visualmarker. The one or more processors can determine a location or positionof the UAV in a three-dimensional coordinate system based on the visualmarker while the UAV is in flight. The marker location inthree-dimensional space can be known, therefore determining the locationof the UAV relative to the marker can result in a determination of thelocation of the UAV in three-dimensional space.

A radar or infrared detector can also aid in determining the location ofthe UAV in three-dimensional space. In the case of radar detection theUAV can emit a radar signal or the marker can emit a radar signal. TheUAV can interpret an emitted or reflected radar signal to determine itsdistance from a surface or obstacle. The UAV can use the detection of asurface or obstacle and compare to a stored floor plan or blueprint todetermine the UAV's location in an environment. Similarly the UAV cancomprise an infrared detector to detect surfaces or obstacles in anenvironment. The UAV can compare detected obstacles and/or surfacesagainst a known environment layout to determine its location in theenvironment.

The location of the UAV in three-dimensional space can be a location inglobal or local three-dimensional space. The UAV can capture an image ofthe visual marker using a vision sensor on board the UAV. The visionsensor on board the UAV can be a vision sensor that is used forsurveillance of the environment or the vision sensor can be a separatevision sensor configured to detect a visual marker. The vision sensormay be a monocular camera, stereo vision camera, radar, sonar, or aninfrared camera. In some cases the vision sensor can capture an image.The image can be stored on or off board the UAV on a memory storagedevice. The vision sensor image can be analyzed by a processor on boardor off board the UAV. The processor can analyze the vision sensor imageto determine the extrinsic properties of the vision sensor. In caseswhere the vision sensor is a camera, the extrinsic properties of thecamera can be the rotation (R) of the camera relative to the visualmarker and the position (t) of the camera relative to the visual marker.The extrinsic properties can be calculated using an N-point perspective(Perspective n points) algorithm which can solve for a camera's positionbased on a correspondence relationship from three dimensional points totwo dimensional points. In this case, the three dimensional points canbe the known location of the visual marker vertices. The two dimensionalpoints can be the points corresponding to the vertices in the visualmarker in one or more camera images. The camera images can represent atwo dimensional projection of the three dimensional visual marker. Thealgorithm can further include variables corresponding to the camera'sintrinsic properties. Intrinsic properties (K) can be parameters thatare specific to the camera's design or components. The intrinsicproperties can change or drift and may be pre-calibrated or recalibratedprior to the beginning of a surveillance mission of a UAV. The camera'sintrinsic properties can be the focal length, principal point, and/orlens distortion. Minimizing error between the known location of theimage vertices in three dimensional space (X_(i)) and the location ofthe vertices in the camera image (x_(i)) with the following equation cansolve for the camera position as a defined by R and t as a function ofthe three dimensional points, two dimensional points, and cameraintrinsic properties (K),

$\min\limits_{R,t}{\sum\limits_{i}{{x_{i} - {{K\left\lbrack R \middle| t \right\rbrack}X_{i}}}}^{2}}$

The perspective n points algorithm can determine the position of acamera or other visual sensor on board a UAV. The location of the cameraon board the UAV can be known relative to the center of mass of the UAVand/or relative to other sensors or components on board the UAV. Thelocation of the camera relative to other components or sensors on boardthe UAV can be combined with the determined location of the camera todetermine the location of the components or sensors on board the UAV.

Determining the location of the UAV based on a determination of a cameraon board the UAV can be used for obstacle avoidance. The visual markercan be adhered to an obstacle, for example, a wall, pillar, door, orother object. The UAV can automatically recognize the obstacle or thevisual marker can have a particular pattern that communicates to the UAVthat the visual marker is adhered to an obstacle. The UAV can captureand analyze the visual marker to determine the distance from theobstacle. In response to the determined distance the UAV can adjust itsheading and/or altitude to avoid the detected obstacle. The UAV cancollect a series of images as it approaches the obstacle to determinethe change in relative position between the UAV and the obstacle as itchanges its heading and/or altitude. An example of an obstacle detectionand avoidance event with a visual marker is shown in FIG. 7. In a firstlocation 701 a UAV 703 can detect a visual marker 704. The visual marker704 can have a pattern that can indicate to the UAV 703 that the visualmarker 704 is adhered to an obstacle 705. The UAV can capture an imageof the visual marker and can analyze the image to determine the locationof the UAV relative to the location of the visual marker. The UAV canadjust its position to achieve a second location 702. After adjustinglocation the UAV 703 can capture a second image of the visual marker 704to determine the new position and to verify that the new position willavoid collision or interaction with the obstacle 705. In cases in whichthe second position is not sufficient to avoid the obstacle the UAV canchange position again and capture another image to confirm that the UAVis on track to avoid the obstacle. The process of capturing images anddetermining the location of the UAV can be repeated until the obstacleis successfully avoided.

In an alternate environment the UAV can automatically detect and avoidobstacles. The UAV can detect obstacles using a vision sensor such as amonocular camera, stereo vision camera, radar, sonar, or an infraredcamera. The processor on board or off board the UAV can be configured tointerpret data from the image sensor to detect an obstacle. The UAV canautomatically adjust its route to avoid a detected obstacle. In somecases the UAV may have prior knowledge of the location of obstacles. Insome cases, these obstacles may be permanent building features such aswalls, beams, or pillars. The UAV may be aware of these permanentbuilding features from an environment map or floor plan stored on amemory storage device on or off board the UAV. The UAV can automaticallyadjust its route to avoid a permanent building feature that may be anobstacle. A UAV can follow a continuous ribbon of visual markers or aseries of consecutive discrete visual markers. While following thevisual marker the UAV may encounter an obstacle. The UAV can avoid theobstacle by rerouting its path around the obstacle and then returning tothe path of the visual marker. In some cases a visual marker can informa UAV that an obstacle is obstructing a path along a ribbon or series ofvisual markers, the visual marker can instruct the UAV to move to avoidthe obstacle and continue along the path.

A UAV can detect a visual marker in an environment. The environment canhave other objects and patterns that the UAV can distinguish from themarker. The UAV can be configured to eliminate false detection events. Afalse detection event may occur when a UAV detects a pattern or objectwith a similar visual structure of a visual marker that may be ofinterest to the UAV. The UAV can be configured to analyze the detectedpattern or object to determine if it is the visual marker of interest ora false visual marker that has been misidentified. In an environment, aUAV can attempt to identify a visual marker by scanning for regions inthe environment with the color and/or shape of the expected visualmarker. The method of scanning for an expected shape and color patternor combination can result in a set of detection regions. The UAV canfurther analyze the detected regions to eliminate false detectionevents. FIG. 8 shows an example of a UAV 801 in an environment. The UAV801 is configured to recognize a visual marker comprising a repeatingpattern of a set of shapes (e.g. square) of a given color or colors. TheUAV may immediately detect a set of possible visual markers 802, 803,and 804. The UAV can further analyze the detected possible visualmarkers 802, 803, and 804 to eliminate false detection events. Thefurther analysis can include processing the visual markers exact color,pattern, shape, and/or size or sizes of relative shapes in the marker.With further analysis the UAV can determine that 802 and 804 are not thevisual markers of interest and therefore represent false detectionevents.

When a UAV detects a visual marker the UAV can determine its locationbased on one or more images of the visual marker and/or receive aninstruction encoded by the visual marker. The instruction can elicit anavigation response by the UAV. The visual marker can include adistinction pattern that can be identified by the UAV. The UAV candetect the visual marker with a visual distinction pattern and cancapture an image of the visual marker using a vision sensor while theUAV is in flight. The image of the visual pattern can be interpreted bya processor on board or off board the UAV. The processor can distinguishand identify the visual pattern from a plurality of possible visualpatterns and determine that the identified visual pattern elicits aresponse from the UAV that is different from a response elicited by aplurality of other possible visual patterns. The processor can initiatethe response elicited by the detected visual pattern.

A visual marker comprising a distinction pattern can have similarelements as a visual marker without a distinction pattern. The visualmarker comprising a distinction pattern can have colored shapes in arepeating patter in addition to having a pattern that can communicate acommand or piece of information to the UAV. The UAV may know thelocation of the visual markers comprising a distinction pattern in anenvironment. In some cases the distinction pattern can be Apriltag, a QRcode, or a bar code. FIG. 9 shows an example of a visual marker with adistinction pattern. A UAV can distinguish from a visual markercomprising a distinction pattern and a marking that looks similar to avisual marker comprising a distinction pattern to avoid false detectionevents.

The distinction pattern can instruct the UAV to perform a mission or toperform a navigation response. A mission or navigation response may betravelling along a path from one place to another, moving a sensor,changing an attitude of the UAV, flying in a specified or particulardirection, flying up or down stairs, adjusting the position of apayload, flying in accordance with preset flight commands, turning asensor on or off, directing a sensor in a specified direction, orcapturing an object. An instruction can include a specified timeinterval that a UAV should wait before performing the mission ornavigation response. An instruction can include a predetermined distancefrom the visual marker that a UAV should be before performing themission or navigation response. A distinction pattern can instruct a UAVto follow a ribbon of visual markers along a path or a series ofdiscrete visual markers along a path. The distinction pattern caninstruct the UAV to be a specified distance from the ribbon while itfollows the ribbon or a series of discrete visual markers along a path.The path can be in an indoor, outdoor, or both indoor and outdoorenvironment. FIG. 10 shows an example of a ribbon of visual markers 1001that can be followed by a UAV 1002. The ribbon of visual markers 1001can be adhered on a surface such that the UAV 1002 can follow the ribbon1001 without colliding with or encountering an obstacle. The UAV can beconfigured to remain at a fixed distance from the visual marker whilefollowing the marker. In an example, the fixed distance can be at least1 cm, 5 cm, 10 cm, 20 cm, 30 cm, 40 cm, 50 cm, 60 cm, 70 cm, 80 cm, 90cm, or 100 cm. The UAV may remain at a fixed orientation with respect tothe visual marker while following the visual marker. For instance, theUAV may remain at a height that is level with the visual marker (e.g.,about 90 degrees). Alternatively, the UAV may fly beneath the visualmarker or above the visual marker by a predetermined number of degreesor within a predetermined range of degrees (e.g., within 30-60 degrees).The UAV can continuously determine its position by analyzing images ofthe visual marker 1001 as it follows the marker to detect and avoidobstacles that are detected by vision sensors or known from a map of theenvironment. In the case shown in FIG. 10 the ribbon of visual markers1001 can lead the UAV 1002 up a staircase. In another case, thedistinction pattern can instruct the UAV to follow a path by instructingthe UAV to move a specified direction and distance. The distinctionpattern can instruct the UAV to move at an angle in three dimensionalspace. The distinction pattern can instruct the UAV to move a specifieddistance. The UAV can calculate the distance traveled by analysis of aspeed and time sensors that may be on board or off board the UAV. FIG.11 shows an example in which the one or more visual markers including adistinction pattern instruction a UAV to move along a path by providingthe UAV with instructions to move discrete directions and distances. InFIG. 11 a UAV 1101 may detect a first visual marker with a distinctionpattern 1102. The distinction pattern 1102 can instruct the UAV to moveupward and to the right a specified distance 1103. The next visualmarker with a distinction pattern 1104 detected by the UAV can instructthe UAV 1101 to move upward at an angle of 30° 1105. The next visualmarker with a distinction pattern 1106 detected by the UAV can instructthe UAV 1101 to move horizontally a specified distance 1107.

In another example, the distinction pattern can inform a UAV about thelayout of an environment. The distinction pattern can inform the UAVthat a feature is present in an environment. A feature can be anopening, a doorway, a corner, an obstacle, an incline, a decline, or anobject. In an example shown in FIG. 12 a visual marker with adistinction pattern 1201 can inform a UAV 1202 that an environment has afeature such as a doorway opening 1203.

The distinction pattern can instruct a UAV to perform a surveillancetask. A surveillance task can involve moving a sensor to capture dataabout a subject of interest. In an example shown in FIG. 13 in a firstposition 1301 a UAV 1302 can have a sensor for collecting surveillancedata in an environment, for example, a camera configured to collectsurveillance data in an environment 1303 with a field of view 1304directed away from a subject of interest 1305. The UAV 1302 can detect avisual marker with a distinction pattern 1306 with a first vision sensorconfigured to detect visual markers. The distinction pattern 1306 caninstruct the UAV 1302 to move or rotate the camera 1303 into a secondposition 1302 such that the camera's field of view 1304 contains thesubject of interest 1305. The distinction pattern can further instructthe UAV to perform a specific surveillance mission on the subject ofinterest 1305. The surveillance mission can include collecting dataabout the subject of interest 1305 from one or more sensors and storingor transmitting the data to an off board computer system.

A UAV may be used for surveillance of an environment. For instance, theUAV may be patrolling the environment. The UAV may collect informationabout the environment using one or more sensors when performingsurveillance of the environment. The UAV may follow a predetermined orregular path for patrolling an environment. Alternatively, the UAV mayfollow a random flight path. The UAV may detect and/or follow a subjectof interest, such as a person, animal, or vehicle. The UAV may respondto one or more detected event within the environment and investigatefurther.

As previously discussed, a UAV may patrol an environment by collectingone or more images within the environment. The UAV may send thecollected images to an external device. The images may be sent off-boardthe UAV. In some instances, the images may be stored in one or morememory units off-board the UAV. The images may be displayed on a displaydevice remote to the UAV. For example, a security monitor may showimages streamed from the UAV. Human viewers may view live-feeds from theUAVs patrolling the area.

In some embodiments the environment may be an indoor environment. Theindoor environment may include the interior of a building or cluster ofbuildings. The indoor environment may include a multi-floor building.The UAV may traverse the multi-floor building by going up and down oneor stairs, or flying straight up or down a stairwell. The UAV may alsobe able to traverse a multi-floor building by using an elevator orflying up or down an elevator shaft. The UAV may also be able totraverse a multi-floor building by exiting a window on one floor andentering through a window on another floor. Some examples of indoorbuildings may include homes, office buildings, shopping malls,hospitals, schools, or any other type of building. The UAV may be usedto patrol the environment.

The visual markers, as described herein, may assist the UAV withpatrolling the environment. The visual markers may orient the UAV withinthe environment. The UAV may be able to determine a UAV positionrelative to the portions of the environment. For example, the UAV may beable to determine which floor the UAV is on. The UAV may also be able totell which room or hallway it is in. The UAV may be able to determineits orientation (e.g., relative to cardinal directions or other types ofdirections). The UAV may be able to determine its position relative toother rooms or hallways of the environment. The UAV may be able todetermine its local or global coordinates. The UAV may be able todetermine its coordinates to a level of accuracy of within about 50 m,30 m, 20 m, 15 m, 10 m, 7 m, 5 m, 3 m, 2 m, 1.5 m, 1 m, 50 cm, 30 cm, 10cm, 5 cm, or 1 cm.

The visual markers may optionally provide instructions to the UAV fornavigation within the environment. For example, visual markers may bedistributed within an environment. A plurality of visual markers may bedistributed throughout the environment. The visual markers may or maynot be positioned so that they are within a line-of-sight of oneanother. During a patrol, a UAV may encounter a first visual marker. TheUAV may identify and respond to instructions that are displayed usingthe visual markers. The instructions may be encoded into the visualpattern of the visual markers, as described elsewhere herein. The firstvisual marker may direct the UAV to fly in a certain manner that willtake the UAV to a second visual marker. When the UAV reaches the secondvisual marker, the UAV may identify and respond to instructions in thesecond visual marker. The second visual marker may direct the UAV to flyin a certain manner that will take the UAV to a third visual marker, andso forth. This may provide point to point navigation for a UAV. In someinstances, the point to point navigation may lead the UAV in a loopwithin the environment to regularly patrol the environment. In someinstances, point to point navigation may permit a UAV to determine itslocation relative to the visual markers. The UAV may determine itsrelative location to the markers without having to determine its globalposition (e.g., global coordinates). The UAV may be able to patrol aregion without calculating or determining its global position.

In some instances, the visual marker may change, or the flightinstructions associated with the visual markers may change. Forinstance, the visual marker may be displayed on a screen that maydynamically change the visual marker. In another instance, the visualmarker may be static, but new flight instructions associated with thevisual marker may be associated with the visual marker. The new flightinstructions may be downloaded to the UAV. In some instances, the UAVmay have multiple sets of flight instructions associated with aparticular visual marker, and may select the associated flightinstruction in accordance with other parameters. For instance, the UAVmay select a set of flight instructions from multiple flightinstructions associated with the visual marker randomly, based on a timeindicated by the clock of the UAV (e.g., if the time is between 3 pm and6 pm, the UAV will follow a first set of instructions and between 6 pmand midnight will follow a second set of instructions), based on sensedenvironmental conditions (e.g., based on the temperature, amount oflight, detected motion, detected sounds), or any other factors. Thedifferent set of flight paths may direct the UAV to arrive at the samevisual marker or different visual markers. For instance, in a firstscenario, the UAV may be directed from a first visual marker to a secondvisual marker via a first flight path. In a second scenario, the UAV maybe directed from the first visual marker to a second visual marker via asecond flight path. Alternatively, in the second scenario, the UAV maybe directed from the first visual marker to a third visual marker. Thismay permit a UAV to follow a different flight path in accordance withdifferent flight instructions. This may provide some changes orunpredictability to the UAV patrol path which may provide increasedeffectiveness of patrol in some instances. This may also permit the UAVpatrol path to change in a manner that may accommodate differentdetected scenarios.

Using visual markers for point to point navigation may permit the UAV toperform surveillance or a patrol in a manner that does not require inputfrom a remote user. The UAV may be able to fly navigate the environmentusing the visual markers without receiving any signal from an externaldevice (e.g., remote controllers, satellite, towers, routers). Sensorson-board the UAV, such as vision sensors, may detect sufficientinformation to permit the UAV to navigate.

As previously discussed, various types of instructions may be visuallyencoded into the visual markers. Flight instructions may be encoded intoor associated with the visual markers. Other instructions, such asinstructions about operation or movement of sensors, instructions tocharge the UAV, or instructions about data to transmit may be included.Examples of instructions regarding sensors may include instructions foroperating a carrier that may cause a sensor to move. The visual markersmay cause a UAV to re-orient a sensor, such as a camera, about one, two,three or more axes. The visual markers may control the sensor directly(e.g., turning the sensor on, off, zooming, changing sensor modes). Thismay aid in the UAV security patrol and permit the UAV to perform datacollection using one or more sensors without requiring instructions froman outside source or device. The sensor instructions or otherinstructions may be provided separately or in combination with flightinstructions from a single visual marker.

A UAV may be able to patrol an environment autonomously orsemi-autonomously with aid of the visual markers. The UAV may be able topatrol the environment autonomously without requiring humanintervention. The visual markers may aid the UAV in navigating theenvironment without requiring input from a human operator. The UAV maydetermine how to fly without requiring a human to manually direct theUAV's flight. The visual marker may permit the UAV to orient itselfwithout the use of GPS or other location sensors that require externalcommunication. The UAV may be able to orient itself within anenvironment without requiring any communications with an externaldevice. The UAV may be able to fly within the environment withoutrequiring communications with an external device. The UAV may be able toconduct surveillance of the environment without requiring communicationswith an external device.

In some embodiments, a human operator may or may not communicate withthe UAV via a remote controller. In some embodiments, the UAV mayoperate autonomously without receiving any input from a human via aremote controller. In some instances, the human may be able to interveneusing a remote controller. For instance, if the human witnesses an eventthrough a live feed captured by the UAV, the human may be able tointervene using the remote controller. In the absence of humanintervention, the UAV may be capable of continuing on its patrol withoutinput from the human operator.

The systems, devices, and methods described herein can be applied to awide variety of movable objects. As previously mentioned, anydescription herein of an aerial vehicle, such as a UAV, may apply to andbe used for any movable object. Any description herein of an aerialvehicle may apply specifically to UAVs. A movable object of the presentdisclosure can be configured to move within any suitable environment,such as in air (e.g., a fixed-wing aircraft, a rotary-wing aircraft, oran aircraft having neither fixed wings nor rotary wings), in water(e.g., a ship or a submarine), on ground (e.g., a motor vehicle, such asa car, truck, bus, van, motorcycle, bicycle; a movable structure orframe such as a stick, fishing pole; or a train), under the ground(e.g., a subway), in space (e.g., a spaceplane, a satellite, or aprobe), or any combination of these environments. The movable object canbe a vehicle, such as a vehicle described elsewhere herein. In someembodiments, the movable object can be carried by a living subject, ortake off from a living subject, such as a human or an animal. Suitableanimals can include avines, canines, felines, equines, bovines, ovines,porcines, delphines, rodents, or insects.

The movable object may be capable of moving freely within theenvironment with respect to six degrees of freedom (e.g., three degreesof freedom in translation and three degrees of freedom in rotation).Alternatively, the movement of the movable object can be constrainedwith respect to one or more degrees of freedom, such as by apredetermined path, track, or orientation. The movement can be actuatedby any suitable actuation mechanism, such as an engine or a motor. Theactuation mechanism of the movable object can be powered by any suitableenergy source, such as electrical energy, magnetic energy, solar energy,wind energy, gravitational energy, chemical energy, nuclear energy, orany suitable combination thereof. The movable object may beself-propelled via a propulsion system, as described elsewhere herein.The propulsion system may optionally run on an energy source, such aselectrical energy, magnetic energy, solar energy, wind energy,gravitational energy, chemical energy, nuclear energy, or any suitablecombination thereof. Alternatively, the movable object may be carried bya living being.

In some instances, the movable object can be an aerial vehicle. Forexample, aerial vehicles may be fixed-wing aircraft (e.g., airplane,gliders), rotary-wing aircraft (e.g., helicopters, rotorcraft), aircrafthaving both fixed wings and rotary wings, or aircraft having neither(e.g., blimps, hot air balloons). An aerial vehicle can beself-propelled, such as self-propelled through the air. A self-propelledaerial vehicle can utilize a propulsion system, such as a propulsionsystem including one or more engines, motors, wheels, axles, magnets,rotors, propellers, blades, nozzles, or any suitable combinationthereof. In some instances, the propulsion system can be used to enablethe movable object to take off from a surface, land on a surface,maintain its current position and/or orientation (e.g., hover), changeorientation, and/or change position.

The movable object can be controlled remotely by a user or controlledlocally by an occupant within or on the movable object. The movableobject may be controlled remotely via an occupant within a separatevehicle. In some embodiments, the movable object is an unmanned movableobject, such as a UAV. An unmanned movable object, such as a UAV, maynot have an occupant onboard the movable object. The movable object canbe controlled by a human or an autonomous control system (e.g., acomputer control system), or any suitable combination thereof. Themovable object can be an autonomous or semi-autonomous robot, such as arobot configured with an artificial intelligence.

The movable object can have any suitable size and/or dimensions. In someembodiments, the movable object may be of a size and/or dimensions tohave a human occupant within or on the vehicle. Alternatively, themovable object may be of size and/or dimensions smaller than thatcapable of having a human occupant within or on the vehicle. The movableobject may be of a size and/or dimensions suitable for being lifted orcarried by a human. Alternatively, the movable object may be larger thana size and/or dimensions suitable for being lifted or carried by ahuman. In some instances, the movable object may have a maximumdimension (e.g., length, width, height, diameter, diagonal) of less thanor equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. Themaximum dimension may be greater than or equal to about: 2 cm, 5 cm, 10cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. For example, the distance betweenshafts of opposite rotors of the movable object may be less than orequal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.Alternatively, the distance between shafts of opposite rotors may begreater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m,or 10 m.

In some embodiments, the movable object may have a volume of less than100 cm×100 cm×100 cm, less than 50 cm×50 cm×30 cm, or less than 5 cm×5cm×3 cm. The total volume of the movable object may be less than orequal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40 cm³, 50cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³, 300 cm³,500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³3, 1 m³, or10 m³. Conversely, the total volume of the movable object may be greaterthan or equal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40cm³, 50 cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³,300 cm³, 500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³,1 m³, or 10 m³.

In some embodiments, the movable object may have a footprint (which mayrefer to the lateral cross-sectional area encompassed by the movableobject) less than or equal to about: 32,000 cm², 20,000 cm², 10,000 cm²,1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm². Conversely, thefootprint may be greater than or equal to about: 32,000 cm², 20,000 cm²,10,000 cm², 1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm².

In some instances, the movable object may weigh no more than 1000 kg.The weight of the movable object may be less than or equal to about:1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg,8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg,or 0.01 kg. Conversely, the weight may be greater than or equal toabout: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1kg, 0.05 kg, or 0.01 kg.

In some embodiments, a movable object may be small relative to a loadcarried by the movable object. The load may include a payload and/or acarrier, as described in further detail elsewhere herein. In someexamples, a ratio of a movable object weight to a load weight may begreater than, less than, or equal to about 1:1. In some instances, aratio of a movable object weight to a load weight may be greater than,less than, or equal to about 1:1. Optionally, a ratio of a carrierweight to a load weight may be greater than, less than, or equal toabout 1:1. When desired, the ratio of an movable object weight to a loadweight may be less than or equal to: 1:2, 1:3, 1:4, 1:5, 1:10, or evenless. Conversely, the ratio of a movable object weight to a load weightcan also be greater than or equal to: 2:1, 3:1, 4:1, 5:1, 10:1, or evengreater.

In some embodiments, the movable object may have low energy consumption.For example, the movable object may use less than about: 5 W/h, 4 W/h, 3W/h, 2 W/h, 1 W/h, or less. In some instances, a carrier of the movableobject may have low energy consumption. For example, the carrier may useless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less. Optionally,a payload of the movable object may have low energy consumption, such asless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less.

FIG. 14 illustrates an unmanned aerial vehicle (UAV) 1400, in accordancewith embodiments of the present disclosure. The UAV may be an example ofa movable object as described herein. The UAV 1400 can include apropulsion system having four rotors 1402, 1404, 1406, and 1408. Anynumber of rotors may be provided (e.g., one, two, three, four, five,six, or more). The rotors, rotor assemblies, or other propulsion systemsof the unmanned aerial vehicle may enable the unmanned aerial vehicle tohover/maintain position, change orientation, and/or change location. Thedistance between shafts of opposite rotors can be any suitable length1410. For example, the length 1410 can be less than or equal to 2 m, orless than equal to 5 m. In some embodiments, the length 1410 can bewithin a range from 40 cm to 1 m, from 10 cm to 2 m, or from 5 cm to 5m. Any description herein of a UAV may apply to a movable object, suchas a movable object of a different type, and vice versa. The UAV may usean assisted takeoff system or method as described herein.

In some embodiments, the movable object can be configured to carry aload. The load can include one or more of passengers, cargo, equipment,instruments, and the like. The load can be provided within a housing.The housing may be separate from a housing of the movable object, or bepart of a housing for a movable object. Alternatively, the load can beprovided with a housing while the movable object does not have ahousing. Alternatively, portions of the load or the entire load can beprovided without a housing. The load can be rigidly fixed relative tothe movable object. Optionally, the load can be movable relative to themovable object (e.g., translatable or rotatable relative to the movableobject). The load can include a payload and/or a carrier, as describedelsewhere herein.

In some embodiments, the movement of the movable object, carrier, andpayload relative to a fixed reference frame (e.g., the surroundingenvironment) and/or to each other, can be controlled by a terminal. Theterminal can be a remote control device at a location distant from themovable object, carrier, and/or payload. The terminal can be disposed onor affixed to a support platform. Alternatively, the terminal can be ahandheld or wearable device. For example, the terminal can include asmartphone, tablet, laptop, computer, glasses, gloves, helmet,microphone, or suitable combinations thereof. The terminal can include auser interface, such as a keyboard, mouse, joystick, touchscreen, ordisplay. Any suitable user input can be used to interact with theterminal, such as manually entered commands, voice control, gesturecontrol, or position control (e.g., via a movement, location or tilt ofthe terminal).

The terminal can be used to control any suitable state of the movableobject, carrier, and/or payload. For example, the terminal can be usedto control the position and/or orientation of the movable object,carrier, and/or payload relative to a fixed reference from and/or toeach other. In some embodiments, the terminal can be used to controlindividual elements of the movable object, carrier, and/or payload, suchas the actuation assembly of the carrier, a sensor of the payload, or anemitter of the payload. The terminal can include a wirelesscommunication device adapted to communicate with one or more of themovable object, carrier, or payload.

The terminal can include a suitable display unit for viewing informationof the movable object, carrier, and/or payload. For example, theterminal can be configured to display information of the movable object,carrier, and/or payload with respect to position, translationalvelocity, translational acceleration, orientation, angular velocity,angular acceleration, or any suitable combinations thereof. In someembodiments, the terminal can display information provided by thepayload, such as data provided by a functional payload (e.g., imagesrecorded by a camera or other image capturing device).

Optionally, the same terminal may both control the movable object,carrier, and/or payload, or a state of the movable object, carrierand/or payload, as well as receive and/or display information from themovable object, carrier and/or payload. For example, a terminal maycontrol the positioning of the payload relative to an environment, whiledisplaying image data captured by the payload, or information about theposition of the payload. Alternatively, different terminals may be usedfor different functions. For example, a first terminal may controlmovement or a state of the movable object, carrier, and/or payload whilea second terminal may receive and/or display information from themovable object, carrier, and/or payload. For example, a first terminalmay be used to control the positioning of the payload relative to anenvironment while a second terminal displays image data captured by thepayload. Various communication modes may be utilized between a movableobject and an integrated terminal that both controls the movable objectand receives data, or between the movable object and multiple terminalsthat both control the movable object and receives data. For example, atleast two different communication modes may be formed between themovable object and the terminal that both controls the movable objectand receives data from the movable object.

FIG. 15 illustrates a movable object 1500 including a carrier 1502 and apayload 1504, in accordance with embodiments. Although the movableobject 1500 is depicted as an aircraft, this depiction is not intendedto be limiting, and any suitable type of movable object can be used, aspreviously described herein. One of skill in the art would appreciatethat any of the embodiments described herein in the context of aircraftsystems can be applied to any suitable movable object (e.g., an UAV). Insome instances, the payload 1504 may be provided on the movable object1500 without requiring the carrier 1502. The movable object 1500 mayinclude propulsion mechanisms 1506, a sensing system 1508, and acommunication system 1510.

The propulsion mechanisms 1506 can include one or more of rotors,propellers, blades, engines, motors, wheels, axles, magnets, or nozzles,as previously described. The movable object may have one or more, two ormore, three or more, or four or more propulsion mechanisms. Thepropulsion mechanisms may all be of the same type. Alternatively, one ormore propulsion mechanisms can be different types of propulsionmechanisms. The propulsion mechanisms 1506 can be mounted on the movableobject 1500 using any suitable means, such as a support element (e.g., adrive shaft) as described elsewhere herein. The propulsion mechanisms1506 can be mounted on any suitable portion of the movable object 1500,such on the top, bottom, front, back, sides, or suitable combinationsthereof.

In some embodiments, the propulsion mechanisms 1506 can enable themovable object 1500 to take off vertically from a surface or landvertically on a surface without requiring any horizontal movement of themovable object 1500 (e.g., without traveling down a runway). Optionally,the propulsion mechanisms 1506 can be operable to permit the movableobject 1500 to hover in the air at a specified position and/ororientation. One or more of the propulsion mechanisms 1500 may becontrolled independently of the other propulsion mechanisms.Alternatively, the propulsion mechanisms 1500 can be configured to becontrolled simultaneously. For example, the movable object 1500 can havemultiple horizontally oriented rotors that can provide lift and/orthrust to the movable object. The multiple horizontally oriented rotorscan be actuated to provide vertical takeoff, vertical landing, andhovering capabilities to the movable object 1400. In some embodiments,one or more of the horizontally oriented rotors may spin in a clockwisedirection, while one or more of the horizontally rotors may spin in acounterclockwise direction. For example, the number of clockwise rotorsmay be equal to the number of counterclockwise rotors. The rotation rateof each of the horizontally oriented rotors can be varied independentlyin order to control the lift and/or thrust produced by each rotor, andthereby adjust the spatial disposition, velocity, and/or acceleration ofthe movable object 1500 (e.g., with respect to up to three degrees oftranslation and up to three degrees of rotation).

The sensing system 1508 can include one or more sensors that may sensethe spatial disposition, velocity, and/or acceleration of the movableobject 1500 (e.g., with respect to up to three degrees of translationand up to three degrees of rotation). The one or more sensors caninclude global positioning system (GPS) sensors, motion sensors,inertial sensors, proximity sensors, or image sensors. The sensing dataprovided by the sensing system 1508 can be used to control the spatialdisposition, velocity, and/or orientation of the movable object 1500(e.g., using a suitable processing unit and/or control module, asdescribed below). Alternatively, the sensing system 1508 can be used toprovide data regarding the environment surrounding the movable object,such as weather conditions, proximity to potential obstacles, locationof geographical features, location of manmade structures, and the like.

The communication system 1510 enables communication with terminal 1512having a communication system 1514 via wireless signals 1516. Thecommunication systems 1510, 1514 may include any number of transmitters,receivers, and/or transceivers suitable for wireless communication. Thecommunication may be one-way communication, such that data can betransmitted in only one direction. For example, one-way communicationmay involve only the movable object 1500 transmitting data to theterminal 1512, or vice-versa. The data may be transmitted from one ormore transmitters of the communication system 1510 to one or morereceivers of the communication system 1512, or vice-versa.Alternatively, the communication may be two-way communication, such thatdata can be transmitted in both directions between the movable object1500 and the terminal 1512. The two-way communication can involvetransmitting data from one or more transmitters of the communicationsystem 1510 to one or more receivers of the communication system 1514,and vice-versa.

In some embodiments, the terminal 1512 can provide control data to oneor more of the movable object 1500, carrier 1502, and payload 1504 andreceive information from one or more of the movable object 1500, carrier1502, and payload 1504 (e.g., position and/or motion information of themovable object, carrier or payload; data sensed by the payload such asimage data captured by a payload camera). In some instances, controldata from the terminal may include instructions for relative positions,movements, actuations, or controls of the movable object, carrier and/orpayload. For example, the control data may result in a modification ofthe location and/or orientation of the movable object (e.g., via controlof the propulsion mechanisms 1506), or a movement of the payload withrespect to the movable object (e.g., via control of the carrier 1502).The control data from the terminal may result in control of the payload,such as control of the operation of a camera or other image capturingdevice (e.g., taking still or moving pictures, zooming in or out,turning on or off, switching imaging modes, change image resolution,changing focus, changing depth of field, changing exposure time,changing viewing angle or field of view). In some instances, thecommunications from the movable object, carrier and/or payload mayinclude information from one or more sensors (e.g., of the sensingsystem 1508 or of the payload 1504). The communications may includesensed information from one or more different types of sensors (e.g.,GPS sensors, motion sensors, inertial sensor, proximity sensors, orimage sensors). Such information may pertain to the position (e.g.,location, orientation), movement, or acceleration of the movable object,carrier and/or payload. Such information from a payload may include datacaptured by the payload or a sensed state of the payload. The controldata provided transmitted by the terminal 1512 can be configured tocontrol a state of one or more of the movable object 1500, carrier 1502,or payload 1504. Alternatively or in combination, the carrier 1502 andpayload 1504 can also each include a communication module configured tocommunicate with terminal 1512, such that the terminal can communicatewith and control each of the movable object 1500, carrier 1502, andpayload 1504 independently.

In some embodiments, the movable object 1500 can be configured tocommunicate with another remote device in addition to the terminal 1512,or instead of the terminal 1512. The terminal 1512 may also beconfigured to communicate with another remote device as well as themovable object 1500. For example, the movable object 1500 and/orterminal 1512 may communicate with another movable object, or a carrieror payload of another movable object. When desired, the remote devicemay be a second terminal or other computing device (e.g., computer,laptop, tablet, smartphone, or other mobile device). The remote devicecan be configured to transmit data to the movable object 1500, receivedata from the movable object 1500, transmit data to the terminal 1512,and/or receive data from the terminal 1512. Optionally, the remotedevice can be connected to the Internet or other telecommunicationsnetwork, such that data received from the movable object 1500 and/orterminal 1512 can be uploaded to a website or server.

FIG. 16 is a schematic illustration by way of block diagram of a system1600 for controlling a movable object, in accordance with embodiments.The system 1600 can be used in combination with any suitable embodimentof the systems, devices, and methods disclosed herein. The system 1600can include a sensing module 1602, processing unit 1604, non-transitorycomputer readable medium 1606, control module 1608, and communicationmodule 1610.

The sensing module 1602 can utilize different types of sensors thatcollect information relating to the movable objects in different ways.Different types of sensors may sense different types of signals orsignals from different sources. For example, the sensors can includeinertial sensors, GPS sensors, proximity sensors (e.g., lidar), orvision/image sensors (e.g., a camera). The sensing module 1602 can beoperatively coupled to a processing unit 1604 having a plurality ofprocessors. In some embodiments, the sensing module can be operativelycoupled to a transmission module 1612 (e.g., a Wi-Fi image transmissionmodule) configured to directly transmit sensing data to a suitableexternal device or system. For example, the transmission module 1612 canbe used to transmit images captured by a camera of the sensing module1602 to a remote terminal.

The processing unit 1604 can have one or more processors, such as aprogrammable processor (e.g., a central processing unit (CPU)). Theprocessing unit 1604 can be operatively coupled to a non-transitorycomputer readable medium 1606. The non-transitory computer readablemedium 1606 can store logic, code, and/or program instructionsexecutable by the processing unit 1604 for performing one or more steps.The non-transitory computer readable medium can include one or morememory units (e.g., removable media or external storage such as an SDcard or random access memory (RAM)). In some embodiments, data from thesensing module 1602 can be directly conveyed to and stored within thememory units of the non-transitory computer readable medium 1606. Thememory units of the non-transitory computer readable medium 1606 canstore logic, code and/or program instructions executable by theprocessing unit 1604 to perform any suitable embodiment of the methodsdescribed herein. For example, the processing unit 1604 can beconfigured to execute instructions causing one or more processors of theprocessing unit 1604 to analyze sensing data produced by the sensingmodule. The memory units can store sensing data from the sensing moduleto be processed by the processing unit 1604. In some embodiments, thememory units of the non-transitory computer readable medium 1606 can beused to store the processing results produced by the processing unit1604.

In some embodiments, the processing unit 1604 can be operatively coupledto a control module 1608 configured to control a state of the movableobject. For example, the control module 1608 can be configured tocontrol the propulsion mechanisms of the movable object to adjust thespatial disposition, velocity, and/or acceleration of the movable objectwith respect to six degrees of freedom. Alternatively or in combination,the control module 1608 can control one or more of a state of a carrier,payload, or sensing module.

The processing unit 1604 can be operatively coupled to a communicationmodule 1510 configured to transmit and/or receive data from one or moreexternal devices (e.g., a terminal, display device, or other remotecontroller). Any suitable means of communication can be used, such aswired communication or wireless communication. For example, thecommunication module 1610 can utilize one or more of local area networks(LAN), wide area networks (WAN), infrared, radio, WiFi, point-to-point(P2P) networks, telecommunication networks, cloud communication, and thelike. Optionally, relay stations, such as towers, satellites, or mobilestations, can be used. Wireless communications can be proximitydependent or proximity independent. In some embodiments, line-of-sightmay or may not be required for communications. The communication module1610 can transmit and/or receive one or more of sensing data from thesensing module 1602, processing results produced by the processing unit1604, predetermined control data, user commands from a terminal orremote controller, and the like.

The components of the system 1600 can be arranged in any suitableconfiguration. For example, one or more of the components of the system1600 can be located on the movable object, carrier, payload, terminal,sensing system, or an additional external device in communication withone or more of the above. Additionally, although FIG. 16 depicts asingle processing unit 1604 and a single non-transitory computerreadable medium 1606, one of skill in the art would appreciate that thisis not intended to be limiting, and that the system 1600 can include aplurality of processing units and/or non-transitory computer readablemedia. In some embodiments, one or more of the plurality of processingunits and/or non-transitory computer readable media can be situated atdifferent locations, such as on the movable object, carrier, payload,terminal, sensing module, additional external device in communicationwith one or more of the above, or suitable combinations thereof, suchthat any suitable aspect of the processing and/or memory functionsperformed by the system 1600 can occur at one or more of theaforementioned locations.

While some embodiments of the present disclosure have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the disclosure. It should beunderstood that various alternatives to the embodiments of thedisclosure described herein may be employed in practicing thedisclosure. It is intended that the following claims define the scope ofthe invention and that methods and structures within the scope of theseclaims and their equivalents be covered thereby.

What is claimed is:
 1. A method of controlling an unmanned aerialvehicle (UAV) comprising: receiving, from one or more vision sensors ofthe UAV, an image including a marker within an environment, the imagebeing captured while the UAV is in flight, and the marker having avisual pattern; identifying, with aid of one or more processors, thevisual pattern to determine a plurality of instructions encoded in thevisual pattern, the plurality of instructions including at least one ofadjusting position of a payload or swapping the payload; andcontrolling, in response to the plurality of instructions, the UAV toperform an action including at least one of adjusting the position ofthe payload or swapping the payload.
 2. The method of claim 1, whereinthe environment is an indoor environment or an environment in whichglobal position system (GPS) signals are not reliable.
 3. The method ofclaim 1, wherein the marker includes a ribbon.
 4. The method of claim 1,wherein the visual pattern includes squares of different colors.
 5. Themethod of claim 1, wherein the visual pattern includes at least one ofAprilTags, QR codes, or bar codes.
 6. The method of claim 1, wherein theaction is independent of flight of the UAV.
 7. The method of claim 1,wherein controlling the UAV to perform the action includes controllingthe UAV to perform the action within at least one of a predeterminedamount of time or a predetermined distance after the visual pattern isidentified.
 8. The method of claim 1, further comprising: calculating adimension of at least a portion of the visual pattern; and determining,with aid of the one or more processors, a location of the UAV within athree-dimensional coordinate system, based on the dimension of the atleast a portion of the visual pattern.
 9. The method of claim 1, furthercomprising: positioning the UAV within the environment by calculating arotation and a position of the UAV relative to the marker using at leastone of intrinsic properties of the one or more vision sensors, apredefined position of the marker, or a set of points in the image. 10.The method of claim 9, wherein the UAV is patrolling the environmentautonomously while the one or more vision sensors capture the image. 11.A system for controlling an unmanned aerial vehicle (UAV) comprising:one or more vision sensors configured to capture an image including amarker within an environment, the image being captured while the UAV isflight, and the marker having a visual pattern; and one or moreprocessors, individually or collectively configured to: identify thevisual pattern to determine a plurality of instructions encoded in thevisual pattern, the plurality of instructions including at least one ofadjusting position of a payload or swapping the payload; and control, inresponse to the plurality of instructions, the UAV to perform an actionincluding at least one of adjusting the position of the payload orswapping the payload.
 12. The system of claim 11, wherein theenvironment is an indoor environment or an environment in which globalposition system (GPS) signals are not reliable.
 13. The system of claim11, wherein the marker includes a ribbon.
 14. The system of claim 11,wherein the visual pattern includes squares of different colors.
 15. Thesystem of claim 11, wherein the visual pattern includes at least one ofAprilTags, QR codes, or bar codes.
 16. The system of claim 11, whereinthe action is independent of flight of the UAV.
 17. The system of claim11, wherein the one or more processors are further configured to controlthe UAV to perform the action within at least one of a predeterminedamount of time or a predetermined distance after the visual pattern isidentified.
 18. The system of claim 11, wherein the one or moreprocessors are further configured to: calculate a dimension of at leasta portion of the visual pattern; and determine a location of the UAVwithin a three-dimensional coordinate system, based on the dimension ofthe at least a portion of the visual pattern.
 19. The system of claim11, wherein the one or more processors are further configured to:position the UAV within the environment by calculating a rotation and aposition of the UAV relative to the marker using at least one ofintrinsic properties of the one or more vision sensors, a predefinedposition of the marker, or a set of points in the image.
 20. The systemof claim 19, wherein the UAV is configured to patrol the environmentautonomously while the one or more vision sensors capture the image.